Prosodic, Spectral and Visual Features for the Discrimination of Prominent and Non-prominent Words

Despite its very high relevance for human communication current spoken dialog systems usually ignore the prosodic variations in the speech signal [1, 2, 3]. In [4] it was shown that speakers use prosodic cues to highlight corrections in a dialog with a machine and that these can be detected using prosodic cues. We extended this idea in [5] to the audio-visual discrimination of prominent from nonprominent words. Visual features have been shown to play an important role for human perception of prosody [6]. In this paper we have a closer look on the information contained in the different features from the acoustic and visual channel. In particular, we investigate the contribution of the visual features, i. e. nose movement and mouth appearance, and spectral features.